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ABSTRACT 

Displays  were  invented  just  in  the  last  century.  The  human  visual  system  evolved  over  millions  of  years.  The  disparity  between 
the  natural  world  “display”  and  that  “sampled”  by  year  2000  technology  is  more  than  a  factor  of  one  million.  Over  1000X  of  this 
disparity  between  the  fidelity  of  current  electronic  displays  and  human  visual  capacity  is  in  2D  resolution  alone  Then  there  is 
true  3D,  which  adds  an  additional  factor  of  over  1000X  The  present  paper  focuses  just  on  the  2D  portion  of  this  grand 
technology  challenge.  Should  a  significant  portion  of  this  gap  be  closed,  say  just  10X  by  2010,  display  technology  can  help 
drive  a  revolution  in  military  affairs.  Warfighter  productivity  must  grow  dramatically,  and  improved  display  technology  systans 
can  create  a  critical  opportunity  to  increase  defense  capability  while  decreasing  crew  sizes. 

Keywords:  displays,  flat  panel  displays,  human  visual  system,  situational  awareness,  warfighter  productivity,  affoidability 

1.  INTRODUCTION 

Defense  must  become  more  capable  yet  crew  sizes  must  shrink  to  achieve  the  acquisition  refoim  objective:  affoidability. 
Warfighter  productivity  must  grow.  The  search  is  on  for  technologies  and  doctrines  that  can  enable  this  mirade.  Display 
technology,  together  with  an  information  dominance  doctrine,  is  poised  to  contribute.  Currently  fielded  weapon  systems  in  the 
year 2000  affond  the  lucky  U.S.  combatant  about  1  million  pixel  visual  display  systems.  The  human  visual  system  (HVS),  as 
discussed  in  Section  3,  is  many  ordeis  of  magnitude  beyond  the  current  level  of  display  technology.  Increasing  the  visual 
bandwidth  to  individual  crew  members  will  enable  the  necessaiy  increase  in  warfighter  productivity. 

2.  DISPLAYS:  BORN  DURING  THE  LAST  CENTURY 

Electronic  information  display  is  a  young  science.  The  television  era  began  in  1927  with  a  technology  demonstration  in  Gennany 
followed  by  commercial  broadcast  initiation  in  New  York  in  1939.  Requirements  from  television,  sensors  and  data  visualization 
have  driven  an  explosion  in  demand  for  electronic  displays  over  the  past  60  years.  Resolution  in  deployed  systems  has  reached 
about  1  million  pixels  per  display  device  in  civil  products.  Transition  to  military  crew  systems  is  spotty.  The  current, 
operational  B-52H  cockpit  shown  in  Figuie  1  is  contrasted  to  the  Rotoreraft  Pilot’s  Associate  (RPA)  advanced  technology 
demonstration  cockpit  in  Figure  2.  The  former  has  just  one  electronic  display,  a  9-in.  cathode  ray  tube  (CRT),  and  dozens  of 
electromechanical  (EM)  instrument  displays.  The  EM  instruments  may  be  though  of  as  electromechanical  computers  with 
integrated  display.  The  RPA  cockpit  comprises  an  array  of  three  12-in.  flat  panel  displays  (FPDs)  implemented  with  active 
matrix  liquid  crystal  display  (AMLCD)  technology.  Improved  electronic  computers  and  flight  control  systems  enable  the 
information  presented  on  the  dozens  of  dedicated  EM  displays  in  the  B-52Hto  be  relegated  to  back-up  formats.  The  result  is  that 
a  single  pilot  can  do  the  work  of  two  in  the  RPA  design.  The  same  paradigm  of  using  improved  display/computer/sensor 
systems  to  increase  warfighter  productivity  can  be  applied  to  all  systems.  Lessons-leamed  on  RPA  may  be  applied  to  advanced 
combat  systems,  including  RAH-66  Comanche,  F-22  Raptor,  Joint  Strife  Fighter  (JSFJ  Army  ground  Future  Combat  Systems 
(FCS),  and  Navy  sea  Future  Naval  Capabilities  (FNC) 


’Paper  citation:  Darrel  G.  Hopper,  “1000  X  difference  between  current  displays  and  capability  of  human  visual  system: 
payoff  potential  for  affordable  defense  systems,”  in  Cockpit  Displays  VII:  Displays  for  Defense  Applications,  Darrel  G. 
Hopper,  Editor,  Proc.  SPIE  4022,  378-389  (2000). 
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Figure  1.  B-52H  pilot  main  instrument  panel  with  one  electronic  display  (9-in.  monochrome  CRT)  plus  some  60 
electromechanical  instruments.  Cockpit  designed  in  1950s,  built  1960-62.  The  Air  Foiee  plans  to  operate B-52s  until  2046. 
Note  closable  screen  for  nuclear  flash  and  synthetic  vision  for  low  level  flight  (LLLTV  or  FL1R  image  displayed  on  CRT) 


Figure  2.  Rotorcraft  Pilot’s  Associate  (RPA)  advanced  technology  demonstiation  aircraft  cockpit  with  three  12-in.  active  matrix 
liquid  crystal  displays  (AMLCDs)  and  virtually  no  electromechanical  instruments.  Cockpit  designed  in  1990s,  built  1998,  flown 
as  non-operational  engineering  program  (one  aircraft)  to  learn  how  to  used  advanced  electronics  to  significantly  reduce  combat 
crew  workload.  The  total  information  carrying  capacity  of  this  research  cockpit  is  twice  that  of  the  F-22A. 
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3.  HUMAN  VISUAL  SYSTEM:  BORN  OVER  1,000,000  CENTURIES  AGO 

Hopper1  demonstrated  that  the  sampling  (pixelation)  of  a  471  steradian  photon  flux  field  passing  through  a  spherical  surface 
may  be  represented  by  the  equation 

Npixel  ((3)  =  5.3465  x  1011 /p2  , 

where  each  pixel  subtends  a  solid  angle  of  pxp,  where  P  is  expressed  in  units  of  arc  seconds.  Note  that  these  are  pixels  (i.e. 
2D  samples),  with  voxelation  (3-D  samples)  ignored  for  purposes  of  the  present  discussion.  The  total  number  of  samples 
(pixels)  needed  to  represent  a  spherical  photon  flux  field  (471  sr)  at  various  acuities,  p,  is  shown  in  Table  1. 


If  one  picks  a  particular  distance,  d,  of  the  display  surface  from  the  design  eye  point,  one  can  compute  the  lineal  pixel  density, 
Dl  .  from  the  equation 

Dl  (pixels/in.)  =  206,270  /  (dp)  , 
or 

Dl  (pixels/in.  |  d=24  in.)  =  8,594.6  /  p 


Areal  pixel  density, 


Da  (pixels/100  sq.  in  |  d=24  in.)  =  100  DL2  , 


and  the  pixel  pitch,  x, 
are  then  readily  derived. 


x  (pixel  pitch  in  pm  |  d=24  in.)  =  25,400  /  DL  =  2.96  P  , 


2 

Devices  with  total  resolution  as  high  as  5,242,880  pixels  have  been  made.”  Tiled  systems  have  been  built  with  aggregate 
resolutions  of  28,000,000  pixels.1  Lineal  pixel  density  for  direct-view  AMLCDs  has  recently  reached  211  pixels  per  in.  (ppi) 
with  120  pm  color  pixels.  Miniature  displays  for  projection  systems  (head-mounted,  hand-held,  large  image)  are  routinely 
made  with  12  pm  pixels.  Display  devices  with  pixel  pitch  as  small  as  6  pm  can  be  made  with  present  technology. 


Table  I.  Number  of  pixels  in  47c  steradians.  Pixel  densities  (lineal  and  areal)  and  pixel  pitch,  at  24  in. 


Acuity  Comment  Pixels  in  471  sr  Pixels  /  Inch  Pixels  /  100  sq.in.  Pixel  Pitch  (pm) 

@  24  in.  @  24  in.  @  24  in. 


100  arc  seconds 

Image  perceivable 

53,465,000 

86 

738,670 

296 

84  arc  seconds 

E-letter,  orientation 

75,772,000 

102 

1,048,200 

249 

50  arc  seconds 

20/20  vision 

213,860,000 

172 

2,954,700 

148 

25  arc  seconds 

2  discs/bars 

855,450,000 

344 

11,819,000 

74 

14  arc  seconds 

Detect  square 

2,727,800,000 

614 

37,687,000 

41 

5  arc  seconds 

Glint,  stars  * 

21,386,000,000 

1,720 

295,470,000 

15 

2  arc  seconds 

Vernier  * 

133,670,000,000 

4,297 

1,846,700,000 

0.5  arc  second 

Line  >  1°  *  2,138,600,000,000 

17,189 

29,547,000,000 

1.5 

*  Real  world  luminance  &  chromaticity  contrast  effects. 
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Perhaps  the  most  widely  know  standard  for  human  visual  acuity  is  20/20.  Table  11  presents  a  comparison  of  20/20  vision  to  the 
scene  generated  by  the  natural  world  and  perceived  by  the  human  visual  system.  It  is  clear  that  20/20  vision  defines  but  a  small 
fraction  of  the  capability  of  an  ideal  display  system.  The  first  century  of  electronic  displays  addressed  little  more  than  20/20 
vision.  The  next  centuries  have  much  potential. 


Table  II.  Comparison  of  20/20  vision  to  natural  world  scene  percdved  by  human  visual  system  (HVS) 


Parameter 

20/20  vision 

Natural  world  /  HVS 

Ambient  Intensity  Range 

None  (fixed  @  ~10  lx) 

10,000,000x  range  (10 3  to  10+5  lx) 

Grayscale 

1  bit  (on/ off) 

20  bits  (all  shading  nuances  included) 

Color 

No  (black/  white) 

Yes  (full,  over  32  million  colors) 

Motion  (video) 

No  (static) 

Yes 

Content 

Letters  (ultra  simple) 

Real  world  images  (ultra  complex) 

Peripheral  Vision 

No 

Yes 

Computer  Latency 

Zero  (static  image) 

Zero  (complex  moving  image  elements) 

4.  ELECTRONIC  DISPLAY  TECHNOLOGY  CIRCA  2000 

Economic  viability  is  used  here  to  categorize  display  technologies  as  first  tier,  second  tier,  and  “wannabe”  (want-to-be).  First  tier 
technologies  support  high  volume  consumer  products;  that  is,  they  are  economically  viable  in  mass  markets.  Second  tier 
technologies  support  low  volume  consumer  products;  that  is,  they  are  economically  viable  in  niche  markets.  Wannabe 
technologies  support  no  products,  or  have  not  been  in  the  market  long  enough  to  establish  economic  viability. 

A  mass  market  is  defined  herein  for  displays  as  involving  tens  of  millions  of  units  per  year  (installed  in  products  or  stand-alone); 
niche  market,  tens  of  thousands  per  year.  One  might  even  define  an  exotic  market  involving  tens  to  hundreds  of  units  per  year. 
All  military  applications  fall  into  the  exotic-to-niche  range  in  tenns  of  unit  volume. 

Specialized  versions  of  consumer  mass  and  niche  technologies  are  used  in  military  products.  Military  performance  specifications 
are  written  in  any  given  procurement  year  to  extract  more  of  the  technology  base  into  its  products  than  consumer  products. 
However,  if  no  civil  product  base  develops,  transition  to  military  products  is  typically  not  affordable.  Thus,  economic  viability 
is  a  goal  of  any  governmental  display  research  program.  The  government’s  role  is  that  of  the  earliest,  highest  risk  investor;  once 
the  potential  has  been  demonstrated,  successful  technology  advances  move  on  toward  market  based  on  years  of  far  larger  private 
investments. 

Figure  3  illustrates  the  status  of  electronic  display  technologies  in  the  year  2000  based  on  these  categories.  There  are  just  two 
first  tier  technologies;  cathode  ray  tube  (CRT)  and  liquid  crystal  display  (LCD)  The  LCDs  include  dichroic  (dLCD),  reflective 
(RLCD),  and  active  matrix  (AMLCD).  For  both  civil  and  military  applications,  the  CRT  still  has  the  largest  installed  base  of 
any  single  technology,  whereas  the  AMLCD  is  the  fasting  growing  technology.  The  AMLCD  technology  now  dominates 
notebook  computers  and  is  moving  into  desktop  monitors.  The  AMLCD  is  the  preferred  technology  in  aircraft  cockpits  (civil 
and  military)  and  most  workstations  (civil  and  military). 

There  are  seven,  second  tier  technologies:  digital  micromirror  devices  (DMD)  alternating  current  gas  plasma  (ACGP);  inorggnic 
electroluminescent  (EL,  TFEL,  AMLL)  vacuum  fluorescent  displays  (VFD)  inorganic  light  emitting  diodes  (LED)  traditional 
macro-electromechanical  displays  (EM);  and  incandescent  light  displays  (ILD).  The  DMD  technology,  which  is  also  called 
Digital  Light  Processing  (DLP)  by  its  sole  producer,  Texas  Instruments,  now  commands  the  professional  presentation  market 
against  miniature  LCD  competition,  and  is  poised  to  take  the  lead  in  digital  cinema.  The  DMD  is  now  included  in  production 
programs  for  military  workstation  displays  and  is  in  development  to  replace  CRTs  in  head-up  displays  and  instrument  panels. 
Plasma  technology  is  succeeding  in  some  television  and  command  room  applications.  Electroluminescent  displays  include  the 
traditional,  thick  film  (EL),  new  low-voltage  thin-film  (TFEL),  and  miniature  active  matrix  (AMEL)  variants.  The  EL  is  often 
used  where  monochrome  or  limited  color  is  sufficient  to  show  symbol  or  video  fonnats;  civil  applications  include  medical 
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instillments  and  trains;  military  applications  include  forward-looking  infrared  (FLIR)  video  and  headmiounted  systems.  Vacuum 
fluoiescent  displays  continue  to  be  used  in  some  automotive  applications.  The  traditional  technologies  of  inorganic  LED,  EM, 
and  incandescent  continue  to  be  used,  especially  in  low-infonnation  display  applications,  where  they  are  cheap,  reliable,  and  fit- 
for-purpose. 

Wannabe  technologies  abound.  Three  that  have  received  considerable  interest  and  investment  over  the  past  few  years  are  solid 
state  laser  projector  (SSLP),  field  emission  displays  (FED);  and  organic  light  emitting  diode  (OLED)  displays.  Prototypes  exist 
for  these  wannabe  technologies,  but  they  have  yet  to  be  incorporated  into  any  product  that  has  succeeded  (i.e.  showed  staying 
power)  in  the  market.  Solid  state  laser  projector  displays  include  both  image-on-screen  designs  and  a  head-mounted  version 
known  as  virtual  retinal  display  (VRD).  There  are  no  products  using  SSLP  technology,  civil  or  military.  Key  technology 
barriers  to  SSLP  displays  include  affondable  solid  state  laseis  with  high  efficiency  at  the  correct  wavelengths  and  higher 
bandwidth  modulation  devices.  The  VRD,  in  particular,  faces  significant  additional  challenges,  such  as  the  public  fear  of  having 
a  laser  beam  directed  through  their  pupil  rather  than  onto  screen.  The  FED  technology  was  picked  in  the  mid-1990s  by  the 
Defense  Advanced  Research  Project  Agency  (DARPA)  to  leap-frog  AMLCD  technology.  The  prediction  in  1994  was  that  FED 
would  command  20%  of  the  AMLCD  market  by  the  year  2000.  The  reality  now  that  it  is  the  year  2000:  the  FED  market  share 
is  zero.  The  technology  challenges  of  bringing  FEDs  to  market  were  significantly  underestimated.  No  products  incorporating  a 
FED  have  showed  staying  power  in  the  market.  New,  presently  unknown,  materials  (such  as  high  efficiency  low-voltage 
phosphors)  and/or  structures  (such  as  anodes  that  do  not  cause  device  failure  by  continually  outgasing  when  operated  at  >10  kV 
for  more  than  a  few  hours)  are  needed  if  FED  is  ever  to  become  a  viable  technology.  Organic  light  emitting  diode  displays  and 
active  matrix  (AMOLED)  displays  have  evolved  very,  very  rapidly  over  the  past  few  years  and  are  poised  in  the  year  2000  to 
become  economically  viable  by  2005.  The  OLED  technology  offeis  the  potential  for  significantly  higher  power  efficiency  than 
available  in  year  2000  AMLCD  technology. 


FIRST  TIER  (economically  viable  in  mass  markets) 

-  Cathode  Ray  Tube  (CRT) 

-  Liquid  Crystal  Display  (AMLCD,  LCD,  RLCD) 
SECOND  TIER  (economically  viable  in  niche  markets) 

-  Digital  Micromirror  Device  (DMD) 

-  Alternating  Current  Gas  Plasma  (ACGP) 

-  Electroluminescent  (EL,  TFEL,  AMEL) 

-  Vacuum  Fluorescent  Display  (VFD) 

-  Inorganic  Light  Emitting  Diodes  (LED) 

-  Macro  Electromechanical  displays  (EM) 

-  Incandescent  light  displays  (ILD) 

WANNABE  (economic  viability  yet  to  be  established) 

-  Solid  State  Laser  Projector  (SSLP,  VRD) 

-  Field  Emission  Display  (FED,  AMFED) 

-  Organic  Light  Emitting  Diode  (OLED,  AMOLED) 

-  Et  Cetera 

Figure  3.  Economic  viability  status  of  electronic  display  technologies. 
Successful  display  devices  enable  the  existence  and  evolution 
of  the  television,  computer,  and  information  industries. 
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5.  VISION  FOR  2010,  2020,  2100,  3000 

The  resolution  of  both  the  natural  world  (nature’s  image  generation  system)  and  the  human  vision  system  exceeds  year  2000 
technology  by  many  orders  of  magnitude.  A  vision  for  2010,  2020,  2100,  and  3000  is  presented  in  Table  III  in  terms  of  the 
metric  of  resolution  per  individual  display  device.  Yet  higher  resolutions  will  be  obtainable,  as  usual,  by  tiling.  Implicit  in 
this  vision  is  the  necessity  to  simultaneously  improve  sensors,  image  generation  computers,  and  support  electronics.  Also 
implicit  is  the  need  to  address  the  materiel  and  structure  issues  needed  to  increase  efficiency  (efficacy  in  lumens  per  Watt)  from 
about  5  lm/W  in  year  2000  to,  for  example,  20  lm/W  by  2010,  and  to  50  lm/W  by  2100.  Specific  power  density  (W/kg)  for 
mobile  power  sources  needs  to  go  up  a  factor  of  10  by  2010  and  100  by  2100.  The  manufacturing  techniques  must  improve  to 
enable  costs  per  megapixel  to  go  down  by  a  factor  of  10  by  2010  and  100  by  2020.  Full  true  3D  display  technology  is 
represented  conceptually  by  the  Holodeck  of  science  fiction  and  is  a  millennial  challenge  to  be  met  by  the  dawn  of  the  31st 
century:  1.3  teravoxels  are  needed. 


Table  III.  Display  vision.  Metric:  resolution  (megapixels  per  display  device).  * 


Year 

Market  Classification  Category  (annual  unit  sales) 

Exotic 

Niche 

Consumer 

(1-100) 

(l-10k) 

(.l-10m+) 

2000 

2001 

5.4  for  computer 

1.3  for  cockpit 

2,  digital  cinema 

0.3  for  cockpit 

1.9  for  personal  computer  (PC) 

2.1  for  high  definition  television  (HDTV) 

2010 

2010 

2 1  for  film  pre-production 

2 1  for  simulator 

21  for  ads,  games 

2  for  cockpit 

2  for  mobile  devices,  furniture  surfaces 

2010 

30  for  digital  IMAX 

20,  web  PCTV 

4  for  web  computer  television  (WCTV)  system 

2020 

30  for  cockpit 

20  for  simulator 

8,  WCTV 

2070 

855,  simulator 

2 1 4  for  home 

15,  WCTV 

2100 

Vernier  display:  2  gigapixels  in 

100  sq.  in.  (6  pm  pixels) 

*  Support  and  image  generation  electronics  challenges  are  implicit  in  the  display  resolution  challenges  listed  here. 
Higher  resolutions  than  shown  obtainable  by  tiling. 


The  year  2000  technology  baseline  is  the  starting  point  for  the  vision.  Fieldable  cockpit  display  technology  in  2000  is 
represented  by  the  F-22A  fighter,  RAH-66  Commanche  helicopter,  and  the  upgraded  C-141/C-130  transport.  Each  pilot  has  650- 
1300  cm"  (100-200  in2)  comprising  2  to  6  color  multifunction  displays  (MFD),  with  one  designated  as  the  primary  flight 
instalment  and  the  rest  providing  other  mission  or  subsystem  information  as  selected.  The  transport  display  systems  use  several 
displays  all  the  same  size  to  ease  the  logistics  support  required  and  to  reduoe  the  number  of  back-up  crews  required  to  support 
combined  C-141/C-130  operations.  Commonality  remains  a  challenge  area:  there  are  no  common  display  sizes  between  F-22A 
and  RAH-66  despite  years  of  Congressional  demands  for  common  avionics  in  these  two  aircraft. 

Digital  cinema,  electronic  sandboxes,  and  integrated  web-PC-TV  units  (WCTV)  require  display  devices  with  higher  resolution. 
The  era  of  digital  cinema  began  as  an  exotic  market  in  1999  with  the  showing  of  the  movie  Starwars  Episode  I  in  four  theatres 
from  a  digital  master  at  1.3  megapixels  per  35mm  film  frame  on  two  technologies,  one  based  on  the  Texas  Instruments  Digital 
Micromirror  Device  (DMD)  Digital  Light  Processing  technology  and  another,  the  Qualcomm/Hughes- JVC  CRT/Liquid 
Crystal  Light  Valve  CineComm  Digital  Cinema  technology.  The  digital  cinema  will  move  from  exotic  to  niche  by  2010  as 
Hollywood  plans  to  see  3,300  digital  cinema  projectors  installed,  each  with  2  megapixel  color  resolution. 
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Digital  cinema,  as  well  as  high  definition  digital  television  (HDTV)  will  drive  volume  up  and  cost  down  for  2  megapixel 
display  devices  and  all  associated  electronics.  The  2.1  megapixel  devices  needed  for  HDTV  will  come  to  define  the  mass 
market  by  20 10. 5  The  TV  standard  beyond  HDTV  may  not  come  until  about  2070  with  mass  production  by  2100.  The 
resolution  for  the  21"'  century  TV  standard  (HDTV  at  2,073,600  pixels)  is  about  7  times  greater  than  20th  century  TV.  Thus, 
the  TV  standard  for  the  22nd  century  should  exceed  15  megapixels  per  display  device.  Mass  markets  evolve  slower  than  niche. 

The  need  for  yet  higher  resolution  electronic  display  devices  in  the  exotic  markets  is  exemplified  by  1MAX.6  The  IMAX 
movie  format  provides  30  megapixels  per  70-mm  frame,  or  some  15  times  the  resolution  of  standard  35-mm  film.  The  image 
presented  can  be  much  larger — the  100  x  80  ft.  screen  at  the  Sony  IMAX  in  New  York  NY  is  the  largest  in  the  western 
hemisphere.  In  3-D  IMAX,  which  was  introduced  in  November  1999  at  the  Smithsonian  in  Washington  DC,  each  eye  gets  a 
30  megapixel  image.  One’s  instantaneous  field  of  view  is  almost  filled,  albeit  with  just  a  fraction,  about  20%,  of  20/20 
resolution.  And  dimness  is  still  an  issue:  15  kW  quartz  lamps  are  required  to  create  enough  light  for  even  a  darkened  theatre. 
Turning  IMAX  digital  will  require  30  megapixel  display  devices,  or  15X  the  resolution  of  HDTV. 

Maps  for  sandboxes  require  33  megapixels/m'  to  meet  the  need  for  digital  devices  to  replace  1  x  1  m  color  printed  maps. 

The  15X  goal  is  realizable  by  2010  for  exotic  and  niche  markets.  Drivers  for  increased  resolution  are  markets  in  entertainment, 
computers,  and  the  internet,  which  will  meld  into  web-based  computer  television  (WCTV)  by  2020. 

Rapid  growth  in  resolution  has  begun.  Creation  of  20  to  30  megapixel  displays  for  simulators,  sandboxes,  cinema,  data 
visualization,  both  at  home  and  office,  will  drive  revolutions  in  both  civil  and  military  affairs,  leading  to  pixel-surfaces  for 
furniture,  walls,  and  rooms  by  2010-2020.  Flexible  and  printable  display  technologies,  on  which  research  has  just  begun,  will 
enable  wallpaper-thin  displays.  Many  should  be  able  to  afford  a  home  “pixel  room”  comprising  214  megapixels  in  six  sides, 
by  2070. 

The  vision  for  the  evolution  of  displays  in  defense  systems  through  2010  is  illustrated  in  Figure  4.  Future  cockpit  design 
concepts  require  the  creation  of  a  panoramic  and  immersive  display  technology  base.  The  opportunity  to  do  so  arises  from 
significant  continuing  investment,  by  both  the  commercial  and  government  sectors,  to  make  the  impossible  possible  for  an  ever- 
expanding  global  commercial  display  industry.  In  this  endeavor  the  military  is  the  beneficiary  of  the  information  age — which  it 
spawned  by  prior  decade  investments  but  which  is  now  driven  by  the  insatiable  appetite  of  civil  markets  for  more  and  better 
visual  communication  and  entertainment  devices.  Our  strategy  is  to  pursue  multiple  technological  approaches:  revolutionary  new 
display  technologies,  groupings  (arrays  or  seamless  tiling)  of  flat  panel  displays,  and  projectors.  We  are  funding,  together  with 
the  Defense  Advanced  Research  Projects  Agency  (DARPA)  several  different  methods  within  each  of  these  approaches.7  The 
engineering  desigp  opportunity  will  be  realized  in  fielded  weapon  systems  only  if  the  operational  community  accepts  the  vision. 
We  expect  that  it  will  because  of  the  environment  in  which  future  warfighters  and  professionals  are  growing  up:  panoramic  video 
games  and  learning  systems,  and  immersive  IMAX  movie  theaters.  Pilots,  sailors,  soldiers,  and  astronauts  not  only  will  accept 
panoramic  and  immersive  displays  in  the  “cockpit”,  they  will  expect  and  demand  them. 
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Figure  4.  Defense  display  vision. 


This  vision  enables  the  cockpit  concept  for  2000,  illustrated  in  Figuie  4,  comprising  a  2000  cm’  (300  in2)  panoramic  direct  view 
head-down  display  (HDD)  system  coupled  with  a  simple  helmet  display  for  off-boresight  cueing  of  smart  munitions.  The  head- 
up  display  (HUD)  is  still  present  as  a  ballistic  munitions  targeting  reticule  unambiguously  and  accurately  aligned  with  the 
airframe.  Deployable  displays  may  be  integrated  cither  side  of  the  HUD. 

By  2010  the  cockpit  canopy  may  be  turned  opaque  to  all  optical  wavelengths  for  mission  segments  flown  within  the  threat 
envelopes  from  air,  ground,  or  space  anti-personnel  laser  weapons  via  a  simple  shade  or  a  complex  display  shell.  A  world  view  is 
created  in  the  closed  cockpit  mode  from  on-board/ off-board  digital  data  bases  and  on/off-board  sensor  suites. 
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6.  DEFENSE  PAYOFF  POTENTIAL 

Affordability  is  defense  policy.  More  military  capability  for  less  money  will  require  increases  in  warfighter  productivity. 
Display  technology  is  poised  to  contribute.  Improvements  in  sensors,  computer  image  generators,  and  displays  are  all  needed 
to  enable  design  of  upgrades  and  new  systems  so  operators  can  reduce  metrics  like  S/target  or  S/ton-mile.  However,  display 
technology  is  the  tall  technology  pole  in  the  year  2000. 

Sensor  systems  already  require  far  more  resolution  and  greyscale  than  currently  fields  display  technology  can  support.  Larger, 
higher  resolution  displays  will  enable  more  complex  threat  environments  to  be  displayed  so  that  pilots  and  combat 
crewmembers  can  establish  situational  awareness  over  ever  more  dense  threat  environments  with  real-time-in-cockpit 
information  presently  reserved  for  intelligence  analysts  and  unavailable  to  tactical  crews  until  after  it  is  too  late. 

Computer  image  generation  capability  is  far  ahead  of  display  technology  too.  Simulator  displays  are  so  low  in  resolution  that 
trainees  are  legally  blind.  Training  and  mission  rehearsal  can  only  be  taken  so  far  in  simulators  because  the  display  systems 
are  so  poor  in  resolution  and  luminance.  The  most  conservative  view  (see  Table  I)  is  that  a  fully  immersive,  4jt  sr,  synthetic 
vision  system  (SVS)  would  require  the  214  megapixels  to  provide  its  occupant  even  just  20/20  resolution  (see  Table  11  for  the 
limitations  of  20/20  vision).  The  year  2000  state-of-the-art  for  an  SVS  is  represented  by  tactical  aircraft  simulators,  for  which 
15.36  megapixels  are  generated  by  tiling  eight  1600x1200  (1,920,000  pixel)  projectors  behind  eight  screens  tessellating  a 

sphere  surrounding  a  trainee  seated  in  a  cockpit.  For  20/20  a  cockpit  SVS  must  provide  some  161  megapixels  in  the  solid 
angle  (80%  of  4jt  sr)  out  of  a  fighter  aircraft  bubble  canopy.  The  SVS  20/20  challenge  is  to  increase  resolution  by  the  ratio  of 
171/15.36,  or  11.  IX.  Thus,  display  devices  of  resolution  11.1  x  1.92  megapixels,  or  21.4  megapixels,  are  required.  A 
challenge  of  10  to  15X  increase  in  display  device  resolution  is  a  reasonable  goal  for  the  display  community  to  achieve  by  the 
year  2010.  Yet  higher  SVS  resolutions  are  wanted,  leaving  much  to  be  done  beyond  2010. 

Future  battlefield  threats  will  require  the  creation  and  fielding  of  panoramic  and  immeisive  “cockpits”  for  air,  land,  sea,  and  space 
systems.  Tactical  aircraft  cockpits  exemplify  the  need.  The  direct  view  now  acquired  by  the  pilot’s  unaided  eyes  looking  out  of 
current  cockpits  might  be  denied  even  during  a  clear  weather  day  by  directed  energy  threats.  Rules  of  engagement,  however, 
require  human-in-the-loop  to  the  last  moment  possible  before  munitions  release  and  during  fly-out  to  ground  targets  to  minimize 
collateral  damage  and  civilian  casualties.  In  addition,  combat  pilots  suffer  from  information  overload  resulting  in  loss  of 
situational  awareness  at  times  when  it  is  most  important:  in  combat.  Beyond  visual  range  objects  are  difficult  to  envision  and 
fit  into  a  total  picture  due  to  small  size  (<50  sq.in.)  of  fielded  cockpit  displays.  Also,  sensor  advancements  provide  ever  higher 
resolution  targeting  imagery  real  time  in  the  cockpit.  These  threats,  together  with  night,  in-weather,  lowdevel  flight  conditions 
are  giving  rise  to  the  need  for  large  head-down  panoramic  displays  in  21st  century  cockpits  and  make  the  case  for  exploration  of 
immeisive  displays. 

As  agile  frequency  lasers  become  more  ubiquitous,  military — and  even  civilian — pilots  might  have  to  fly  some  flight  segments 
without  looking  out  of  the  cockpit.  The  canopy  would  be  closed  by  curtains  or  by  an  electrically  controlled  opaquing 
layer — only  during  these  times.  Then  a  synthetically  generated  view  of  the  real  world  would  be  created.  The  control  and  display 
system  might  logically  evolve  as  an  extension  of  present  day  night/in-weather  instrumented  flight  systems.  A  more  extensive  in¬ 
cockpit  display  suite  may  become  necessary  to  survival  and  mission  success.  Such  a  system  might  include  a  4000  cm’  (600  in’) 
head-down  color  multi-function  display  that  would  be  viewable  in  sunlight  or  starlight,  plus  a  helmet-mounted  cueing  system. 
The  opaqued  cockpit  might  include  a  closrible  canopy  display  to  provide  simulated  vision  over  the  full  field  of  view,  a  view 
denied  episodically  during  some  missions  by  external  conditions.  Ideally  the  individual  display  units  comprising  the  closrible 
display  system  would  be  physically  redundant  yet  appear  seamless. 

Before  going  on  it  is  useful  to  note  that  protecting  pilots  from  bright  light  and  providing  them  with  synthetic  vision  in  lieu  of  a 
view  out  the  cockpit  window  is  not  a  new  problem.  The  B-52  cockpit  shown  in  Figure  1  had  window  shades  for  nuclear  flash 
protection  and  presently  has  a  9-inch  display  to  show  FLIR/LLLTV  to  the  pilots  while  flying  at  night  at  very  low  altitude. 
However,  the  new  thmstto  create  a  flexible  display  technology  base,  just  begun  by  DARPA  might  one  day  (about  2010)  enable 
these  closrible  screens,  initially  just  for  protection  against  nuclear  flash  or  laseis,  to  become  additional  display  surfaces. 
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A  program  of  studies  conducted  in  the  early  1990s  demonstrated  an  effective  way  to  increase  fighter  pilot  productivity.  The 
approach  in  this  program,  entitled  "Panoramic  Cockpit  Control  and  Display  System  (PCCADS),"  is  to  provide  the  pilot  with 
large  area  displays  and  a  helmet-mounted  off-axis  target-acquisition  weapon-targeting  system.  There  were  two  projects,  one 
focused  nearterm,  one  far.  The  PCCADS  2000  cockpit  was  designed  to  be  realizable  with  1995  technology  with  production  by 
2000  and  featured  a  25  cm  (10  in.)  square  tactical  situation  display  and  two  15  cm  (6  in.)  squaro  secondary  multifunction  displays 
on  either  side.  All  displays  were  full  color  capable  with  a  total  area  of  1110  cm'  (172  in').  The  test  mission  was  for  an  F-15E. 
A  28%  increase  in  exchange  ratio  was  achieved  versus  the  standard  F-15E  cockpit.  An  18%  increase  was  observed  for  the 
addition  of  helmet  cueing  to  the  F-15E  baseline  cockpit.  Coupling  this  large  display  with  a  helmet -mounted  cueing  system  for 
off  axis  target  acquisition  resulted  in  a  45%  increase.  The  F-22A  Raptor  will  realize  the  PCCADS  2000  concept  in  a  production 
cockpit  comprising  six  flat  panel  AMLCDs  with  an  aggregate  resolution  of  1.35  megapixels  at  5-bit  greyscale  in  1290  cm' 
(201  in')  plus  helmet  cueing.  Beyond  the  PCCADS  2000  cockpit  was  PCCADS:  a  2000  cm2  (300  in')  seamless  head  down 
display  system  implemented  in  a  physically  redundant  fashion.  This  research  demonstrated  pilot  productivity  payoff  from  bigger, 
better  displays. 

A  super-panoramic  cockpit  (SPC)  with  features  beyond  PCCADS  is  illustrated  in  Figuie  5.  The  SPC  concept  is  PCCADS  plus 
(a)  left  and  right  curved  instrument  panel  “wing”  displays,  (b)  flip-up  FPDs,  and  (c)  closable  canopy  display.  A  closable  curtain 
inside  of  the  canopy  in  the  near  term  gives  way  to  a  flexible  canopy  display  in  the  far  term.  Stowable  FPDs  or  projection  screens 
are  deployable  either  side  of  the  HUD. 


SUPEKPANORAM1C  COCKPIT  WITH  CLOSABLE  OPAQUE  LAYER 
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Figure  5.  Concept  for  2010:  Super  Panoramic  Cockpit  Pius 
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The  2020  vision  is  an  encapsulated  cockpit  as  illustrated  in  Figure  6.  Concepts  beyond  2000  will  switch  from  the  outside¬ 
looking-in  (OLI)  to  the  inside-looking-out  (ILO)  approach.  One  develops  the  sense  that  one  is  inside  an  artificially  generated 
scene;  or  world,  when  the  IFOV  exceeds  about  100°  x  50°.  The  pilot  may  have  no  windows  and  their  cabin  may  be  a 
self-contained  spheroid  embedded  within  the  aircraft  or,  possibly,  elsewhere.  This  display  system  might  be  that  of  a  present-day 
trainer/simulator-only  far,  far  better  in  luminance  range,  color  and  resolution.  The  pilot  has  the  option  of  retaining  or  selectively 
removing  real  world  visual  effects  of  weather  and  night  The  2020  vision  includes  actual  views  from  not  only  ownship,  but  also 
from  a  variety  of  other  platforms.  The  capsule  is  a  node  in  the  battle  network. 


Figure  6.  Concept  for  2020:  Immersive  Cockpit. 
Encapsulated  crewspace  realized  via  combination 
of  direct-view,  projection  and  head-mounted  displays. 
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The  grand  challenge  for  display  technology  is  to  close  the  fantastic  106  gap  between  devices  and  the  human  visual  system. 
Predicting  the  pace  of  technology  is  difficult.  Wilber  Wright  predicted  in  1909  that  “No  airplane  will  ever  fly  from  New 
York  to  Paris”  because  the  motor  would  not  take  the  stress  and  the  airplane  “will  always  be  a  special  messenger,  never  a  load 
carrier”.  Within  20  years  Wright’s  predictions  were  proven  far,  far  too  conservative.  Yet,  science  fiction  does  not  readily 
become  science  fact  for  mass  markets  like  TV.  Both  of  these  concerns  have  been  considered  in  establishing  a  vision  of  how 
fast  the  1000X  difference  between  current  displays  can  be  closed  to  enable  affordable  defense  systems.  A  goal  of  10X  device 
resolution  increase  by  2010  will  enable  a  revolution  in  military  affairs  as  it  will  drive  warfighter  productivity. 
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